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MULTIPLE DESCRIPTION CODING VIDEO TRANSMISSION USING 
DE-INTERLACING MECHANISMS 

The present invention relates generally to the transmission of video sequences (20) 
over a netw^ork. More particularly, the present invention relates to methods of transmitting 
and receiving robust video over error prone channels of a network. 

As communication over wireless systems and the Internet has become more 
5 predominant, ways to reliably send and receive video streams over such networks have been 
developed. Multiple description coding (MDC) is one technique that has been shown to be 
effective for such communications. MDC involves the separation of video streams into 
multiple correlated coded representations, or descriptions, of the video signal, and 
transmission of the representations on separate channels for error resilience. With this 

10 technique, an acceptable signal quality can be obtained using a subset of the descriptions, 
with the quality improving as the number of subsets received increases. One way of 
splitting the video streams is by separating the stream into odd and even frames and then 
coding the streams independently. When one of the streams is received, it can be decoded 
at half the frame rate. Due to the correlated nature of the video streams, intermediate 

15 frames that may become lost during transmission may be recovered using motion 
compensated error concealment techniques. 

Examples of techniques using motion compensated error concealment are Multiple 
State Encoding, Video Redundancy Coding (VRC) and Multiple Description Motion 
Compensation (MDMC). Generally, a Multiple State Encoding system includes an encoder 

20 that receives a video stream and encodes the video into independently decodable packet 
streams by employing multiple state encoding with multiple states, and a receiver that 
receives and combines the multiple streams into a single stream and decodes the received 
stream to reconstruct the original video stream. 

Referring to Fig. 1, a simplified block diagram of an existing VRC encoder is 

25 shown. Here, the video signal, consisting of a series of frames 10, is to be transmitted. The 
odd 10a and even 10b frames are separated and encoded using two standardized coders 12, 
and then the descriptions are transmitted over the network. In the event that a frame 10 is 
corrupted or lost in the transmission, the frame 1 0 can be reconstructed using a standardized 
decoder by interpolation from neighboring frames of the other data stream or description. 

30 Hence, the reconstruction is performed using purely temporal information, as no spatial 
information is available. Additionally, due to the signal being split and encoded, the 
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temporal distance between the frames is relatively large, which will decrease the coding 
efficiency. 

Implementing the MDMC technique will provide a system with better coding 
efficiency. Here, non-standardized coders/decoders are employed. Using MDMC two 
5 descriptions can be generated, where each includes coded information for alternating 

frames. Temporal predictors are used that allow the encoder to use both past even and odd 
frames while encoding. This creates a mismatch between the encoder and the decoder when 
only one description is received by the decoder at the receiver side of the network. This 
mismatch error is explicitly encoded to overcome the mismatch transmission error. With 

10 MDMC the coding parameters, such as temporal filters, can be adjusted to a desired trade- 
off between coding efficiency and error resilience. Thus, an MDMC system provides 
reasonable flexibility between coding eJBRciency and error resilience. 

While the use* of MDMC coding provides coding efficiency benefits over VRC 
schemes, MDMC coding requires non-standardized coders/decoders that are not present in ' 

1 5 existing video display equipment. Thus, there exists a need for a way to transmit video 
information over error prone networks in an efficient and error resilient manner using 
existing equipment. 

The present invention satisfies the above described need. In accordance with 
principles of the present invention, an improved method for transmitting and receiving 

20 video signals is provided. At a transmitter side of a network, a progressive video sequence 
(20) is interlaced and the interlaced sequence is split into multiple streams. The multiple 
streams are encoded using encoders and then the streams are transmitted over independent 
channels of the network. Preferably the sequence is split into two streams of signals. At the 
receiver side of the network, the two streams are received and separately decoded. If there 

25 were no transmission errors, the decoded streams are regrouped into the original progressive 
video sequence (20). If however, there were transmission errors, de-interlacing algorithms 
are used to reconstruct the corrupted stream of signals. 

The novel features of this invention, as well as the invention itself, both as to its 
structure and its operation, will be best understood from the accompanying drawings, taken 

30 in conjunction with the accompanying description, in which similar reference characters 
refer to similar parts, and in which: 

Fig. 1 is a simplified block diagram of a prior art VRC encoder; 
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Fig. 2 is a simplified diagram illustrating the how progressive video signals are 
currently transmitted over networks; 

Figs. 3A and 3B are simplified block diagrams illustrating a transmitter and receiver 
for communicating progressive video signals over networks in accordance with principles 
5 of the present invention; 

Fig. 4 shows a representation of interlaced video signals in accordance with 
principles of the present invention; and 

Fig. 5 shows the reconstruction of a lost or corrupted video image in accordance 
with principles of the present invention. 
10 As most video images are now in digital format such as on DVDs, the video is often 

stored in progressive format. Fig. 2 shows a simplified block diagram representation of a 
video sequence 20, consisting of progressive pictures A, B, C, being encoded with a 
standardized video encoder 22, such as an MPEG-2 or MPEG-4 encoder, for transmission 
over a network. 

15 Referring now to Figs. 3 A, 3B and 4, a device and method of transmitting the same 

video sequence 20 according to principles of the present invention will now be described. 
Each of the progressive pictures A, B and C of the video sequence 20 consists of odd and 
even fields (e.g. Ao, Ae, Bo, Be, Co, Ce). At the transmitter 300, the video signal 20 is 
interlaced with an interlacer 302. Interlacing involves vertically subsampling the pictures 

20 with a factor of two, by separating the odd scanning lines and the even scanning lines 
separately. This results in pictures containing only the odd scanning lines, hereinafter 
referred to the odd fields, and picture containing only the even scanning lines, hereinafter 
referred to the even fields, as shown in Fig. 4. Here, it is important to note that none of the 
original scanning lines is lost, i.e., the total number of scanning lines before and after the 

25 above described interlacing is performed, is identical. The interlaced signal 30 is then 

separated into a video stream of odd fields 32 and even fields 34. The video streams of odd 
and even fields are separately encoded with standardized MPEG-2/4 encoders 304, 306, 
creating two descriptions each having their own prediction vectors and residues after the 
encoding. The encoded descriptions are then transmitted over independent channels 308, 

30 3 1 0 to a receiver 320. 

At the receiver 320, both streams of encoded signals can be decoded using 
standardized MPEG-2/4 decoders 322, 324. If the streams are received and decoded with 
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no transmission errors, the decoded streams are regrouped to form the original progressive 
video sequence 20. 

However, if during transmission one of the streams got corrupted, or a field in the 
stream was lost, the present invention provides for a way to estimate the corrupt or missing 
5 information from the information that is correctly received. In accordance with principles 
of the present invention, a deinterlacer 326, employing standard de-interlacing techniques, 
can be used to estimate the corrupt or missing information. In general, de- interlacing can be 
viewed as the reverse process of interlacing. De- interlacing doubles the vertical resolution 
with respect to the interlaced video, and is also aimed at removing subsampling artifacts 

10 caused by the interlaced sampling of the video. For background information on de- 
interlacing, an overview and examples of de-interlacing techniques are described in G. de 
Haan and E.B. Sellers, "De-interlacing: an overview," Proceedings of the IEEE, 86(9): 
1839-1857, September 1998; and E.B. Sellers and G. de Haan, "De-interlacing: A key 
technology for scan rate conversion," Elsevier Science book series Advances in Image 

15 Communications, vol. 9, September 2000. Many de- interlacing techniques currently exist, 
and many new ones are also being developed. A particular de-interlacing technique that can 
be used in accordance with the present invention is foimd in commonly owned US Patent 
6,618,094 entitled "De-Interlacing Image Signals," the contents of which is herein 
incorporated by reference in its entirety. Using this technique, at least three de-interlacing 

20 algorithms are applied to the video signal to obtain three de -interlaced video signals, where 
different majorities of the algorithms have certain strengths and no majority of the 
algorithms copies a single spatio-temporally neighboring pixel to the interpolated position. 
An order statistical filter may then be used to obtain a single output signal from the tiiree 
de-interlaced signals. 

25 Fig. 5 shows an example of how de- interlacing can be used in accordance with the 

present invention to reconstruct a non-received field of a picture. In this example, the odd 
field of picture B, Bo, was lost during the transmission. A de-interlacer is capable of 
reconstructing the lost Bo field based on information in the well received Be field and the 
regrouped A picture. Here, the de-interlacer capable of performing this reconstruction is a 

30 vertical temporal median filter that inherently switches betvv^een field insertion and line 
repetition. The interpolated samples are formed as the median value of the vertical 
neighbors and the temporal neighbor in the previous field. Thus, the missing field is 
interpolated from both spatial and temporal information. 
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While the above described preferred embodiment separates the video sequence into 
two streams of odd and even fields and generates two descriptions which are transmitted 
over two independent channels, other variations are possible. For instance, those skilled in 
the art would recognize that the video sequence can be split into a plurality of multiple 
5 streams, and the sequence can be split using other parameters. 

The present invention provides advantages over the existing video transmission 
methods using multiple description coding. As described above, the method in accordance 
with the present invention uses de- interlacing techniques to reconstruct the progressive 
video in the event that an encoded field was corrupted during transmission. In this manner, 

10 both spatial as well as temporal information is used (in case of so-called motion adaptive or 
motion-compensated de-interlacing, whereas only spatial information is used in, for 
example, directional de-interlacers), thus a high quality reconstruction of the video can be 
achieved even when an unreliable transmission channel is used. Additionally, the error 
concealment can be achieved using existing post processing techniques in existing 

1 5 standardized decoders. 

While the particular embodiments of the multiple description coding scheme as 
illustrated herein are fully capable of satisfying the needs and providing the advantages 
herein before stated, it is to be understood that many changes in construction and circuitry 
and widely differing embodiments and applications of the invention will suggest themselves 

20 without departure from the spirit and scope of the invention. The disclosures and the 

description herein are purely illustrative and are not intended to be in any sense limiting. 
Rather, the invention is to cover all modifications, equivalents, and alternatives falling 
within the spirit and scope of the invention as defined by the appended claims. 
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